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О компании 
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О X5 Технологии 


Х5 Технологии — бизнес-единица Х5 Retail 
Group, которая создаёт комплексные 
цифровые решения для бизнеса и торговых сетей 
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НЕМНОГО СКУЧНЫХ ВВОДНЫХ 


• Назначение 
• Возможности 
° Альтернативы 


• Основные понятия и архитектура 


ПОЧЕМУ ПОЯВИЛСЯ HELM? 
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ПРОЕКТЫ / МИКРОСЕРВИСЫ 


Project 1 


ПРОЕКТЫ / МИКРОСЕРВИСЫ 


Test Stage 
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Ргојесї 2 


ПРОЕКТЫ / МИКРОСЕРВИСЫ 


Stage 
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Project 3 


ПРОЕКТЫ / МИКРОСЕРВИСЫ 


Project М 


HELM — возможности 


• Простая текстовая шаблонизация (go templates + 
Sprig framework) 


° Управление релизами (установка, апгрейд, откат, 
удаление) 


• Tracking выката 
° 3-way merge 


° Hooks & plugins 


• Развитое community 


° Множество готовых chart'oB для всего 
https://artifacthub.io 


artifacthub.io 
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РЕШЕНИЯ ДЛЯ ДЕПЛОЯ B KUBERNETES 


• Kustomize 

• Kubernetes operators 

° Ansible 

• Terraform 

• Jsonnet 
Tanka (by Grafana) 
Qbeq (by Splunk) 
Ksonnet 


НЕДОСТАТКИ HELM 


° Только строковая шаблонизация 
• Сложное расширение 


• Низкий уровень абстракции 


• Потенциальная за путанность 


N ___А 
GrafanaTanka 


Community Call 


НЕДОСТАТКИ HELM? 
Helm Support! 


T HELM 


АРХИТЕКТУРА HELM3 


Е . Kubernetes cluster 
Chart’s repository 


ReleaseN+1 


ОСНОВНЫЕ ПОНЯТИЯ 


• Chart 
• Chart repository 
• Release 


• Values 


HELM CHART ./mychart 


— Chart.yaml 
helm create [chartname] [— charts 
— templates 
I— NOTES.txt 
l— _helpers.tpl 
|-- deployment.yaml 
— hpa.yaml 
— ingress.yaml 
— service.yaml 
[— serviceaccount.yaml 
L— tests 
L— test-connection.yaml 
L— values.yaml 


Guide: https://helm.sh/docs/topics/charts/ 


HELM CHART . /mychart 
I— Chart.yaml 
helm create [chartname] [— charts 
— templates 
— NOTES.txt 
l— _helpers.tpl 
|-- deployment.yaml 
— hpa.yaml 
— ingress.yaml 
— service.yaml 
[— serviceaccount.yaml 
L— tests 
L— test-connection.yaml 
L— values.yaml 


Guide: https://helm.sh/docs/topics/charts/ 


Chart.yaml — minio 


apiVersion: vl 
appVersion: master 
description: High Performance, Kubernetes Native Object Storage 
home: https://min.io 
icon: https://min.io/resources/img/Llogo/MINIO wordmark.png 
keywords: 
- storage 
- object-storage 
- S3 
maintainers: 
- email: dev@minto. io 
name: MinIO, Inc 
name: minio 
sources: 
- https://github.com/minio/minio 
version: 8.0.10 
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Chart.yaml — minio 


apiVersion: vl 
appVersion: master 
description: High Performance, Kubernetes Native Object Storage 
home: https://min.io 
icon: https://min.io/resources/img/Llogo/MINIO wordmark.png 
keywords: 
- storage 
- object-storage 
- S3 
maintainers: 
- email: dev@minio.to 
name: MinIO, Inc 
name: minio 
sources: 
- https://github.com/minio/minio 
version: 8.0.10 


HELM CHART REPOSITORY 


https://example.com/charts 
JFrog 
O ARTIFACTORY 


charts/ 
| 
|- index.yaml 


| CHARTMUSEUM 
|- alpine-0.1.2.tgz 


| 
|- alpine-0.1.2.tgz.prov 


Ze 
— W 


S 


Guide: https://helm.sh/docs/topics/chart_repository/ 


HELM RELEASE, HELM VALUES 
• Chart + Values = Release 


~$ helm -n sum-demand-forecast-develop List 
NAME NAMESPACE REVISION 
sum-demand-forecast-develop sum-demand-forecast-develop 36 


Helm storage backends: configmap / secret / SQL 


~$ kubectl -n sum-demand-forecast-develop get secret 

NAME TYPE 
sh.helm.release.vl.sum-demand-forecast-develop.v33 helm.sh/release.vl 
sh.helm. release. vl.sum-demand-forecast-develop.v35 helm.sh/release.vl 
sh. helm. release. vl.sum-demand-forecast-develop.v36 helm.sh/release.vl 


UPDATED 


2021-04-30 15:29:47.339850651 +0000 


DATA 
1 
1 
1 


AGE 
39d 
21d 
16h 


MUSTHAVE УАМЕ-ДЕВЕЛОПЕРА 


1. Кастомизация приложения в разрезе 
окружений 


Переиспользвание кода 


Управление блоками кода по условию 


Модульная разработка 

Трекинг выката 

Установка очерёдности запуска подов 
Отладка 


Релизный цикл 


ез аз п а ы ы 


=> => 
Расширение функциональности HELM 


YAGNI KISS 


MUSTHAVE УАМЕ-ДЕВЕЛОПЕРА 


1. Кастомизация приложения в разрезе 
окружений 


Переиспользвание кода 


Управление блоками кода по условию 


Модульная разработка 

Трекинг выката 

Установка очерёдности запуска подов 
Отладка 


Релизный цикл 


ез аз п а ы ы 


=> => 
Расширение функциональности HELM 


YAGNI KISS 


РАБОЧИЕ ОКРУЖЕНИЯ 
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(5) Caching service 


(7) Full text search 
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Copy of data 


HELM CHART 


./mychart 
[— Chart.yaml 


[— charts 
— templates 
I— NOTES. txt 
I— _helpers.tpl 
|— deployment.yaml 
I— hpa.yaml 
I— ingress.yaml 
I— service.yaml 
| serviceaccount.yaml 
L— tests 
L— test-connection.yaml 
L— values.yaml 


HELM CHART 


./mychart 
[— Chart.yaml 
[— charts 
— templates 
I— NOTES. txt 
I— _helpers.tpl 
L— deployment.yaml 
I— hpa.yaml 
— ingress.yaml 
I— service.yaml 
[— serviceaccount.yaml 
L— tests 
L— test-connection.yaml 
L— values.yaml 


values.yaml – ОПИСАНИЕ ЗНАЧЕНИЙ 


nodel: 
value: "definition" 
subNodel: 
VALUE: "DEFINITION" 
subNode21: 
VALUE: SOME 
VALUE1: Thing 
subNode22: 
VALUE: SOME 
VALUE1: Thing 
node2: 
value: "definition" 
subNodel: 
- "listNamel" 
settingl: "settingValuel" 
- "listName2" 
settingl: "settingValuel" 


values.yaml — ОПИСАНИЕ ЗНАЧЕНИЙ 


app: 
databaseurl: postgres://myapp:mypass@postgres :5432/backend 
storage: 
persistence: "true" 
storageclass: 
aws: gp2 
gcp: standard 
persistentVolumeSize: 
aws: 10G 
gcp: 20G 
alerts: 
from: "info.producttonecompany.local" 
receivers: 


- "театасотрапу. Local" 
host: "“smtp.company. local" 
- "supportesubcompany.com" 
host: "smtp.subcompany.com" 


ИСПОЛЬЗОВАНИЕ VALUES 


contatners: 
- name: application 
image: registry.company.local/myapplication:0.60.1 
env: 
- name: DBURL 
value: 44 .Values.app.databaseurl }} 
- name: SAVECACHE 
value: {{ .Values.app.storage.persistence }} 
- name: NOTIFYFROM 
value: {{ .Values.alerts.from }} 
ports: 
- name: http 
contatnerPort: 8080 


ИСПОЛЬЗОВАНИЕ VALUES 


contatners: 
- name: application 
image: registry.company.local/myapplication:0.0.1 
env: 
- name: DBURL 
value: {{ .Values.app.databaseurl }} = 
- name: SAVECACHE 
value: {{ .Values.app.storage.persistence |) mm 
- name: NOTIFYFROM 
value: {{ .Values.alerts.from }} mm 
ports: 
- name: http 
contatnerPort: 8080 


СПОСОБЫ ПЕРЕДАЧИ VALUES 


helm upgrade --install -п ci-namespace ci-release path/to/chart \ 


--set global.env=develop 


СПОСОБЫ ПЕРЕДАЧИ VALUES 


helm upgrade --install -п ci-namespace ci-release path/to/chart \ 
--set global.env=develop \ 
--set app.databaseurl-$POSTGRESQL PASSWORD DEV 


СПОСОБЫ ПЕРЕДАЧИ VALUES 


helm upgrade --install -п ci-namespace ci-release path/to/chart \ 
--set global.env=develop \ 
--set app.databaseurl=$POSTGRESQL PASSWORD DEV \ 
--set app.cloud=aws,app.storage.storageclass.aws=gp3 


СПОСОБЫ ПЕРЕДАЧИ VALUES 


helm upgrade --install -п ci-namespace ci-release path/to/chart \ 
--set global.env=develop \ 
--set app.databaseurl=$POSTGRESQL PASSWORD DEV \ 
--set app.cloud=aws,app.storage.storageclass.aws=gp3 \ 


--set alerts.receivers={“dev@company. local” ,”dev@subcompany .com”) 


СПОСОБЫ ПЕРЕДАЧИ VALUES 


helm upgrade --install -п ci-namespace ci-release path/to/chart \ 
--set global.env=develop \ 
--set app.databaseurl=$POSTGRESQL PASSWORD DEV \ 
--set app.cloud=aws,app.storage.storageclass.aws=gp3 \ 
--set alerts.receivers-(“devâcompany.local”,”devâêesubcompany.com”) N 
--set alerts.receivers[0].hostz"smtpdev.subcompany.com" 


СПОСОБЫ ПЕРЕДАЧИ VALUES 


helm upgrade --install -п ci-namespace ci-release path/to/chart \ 
--set global.env=develop \ 
--set app.databaseurl=$POSTGRESQL PASSWORD DEV \ 
--set app.cloud=aws,app.storage.storageclass.aws=gp3 \ 
--set alerts.receivers-(“devâcompany.local”,”devâêesubcompany.com”) N 
--set alerts.receivers[0].host-"smtpdev.subcompany.com" \ 
--values-./path/to/app.dev.yaml 


СПОСОБЫ ПЕРЕДАЧИ VALUES 


helm upgrade --install -п ci-namespace ci-release path/to/chart \ 
--set global.env=develop \ 
--set app.databaseurl=$POSTGRESQL PASSWORD DEV \ 
--set app.cloud=aws,app.storage.storageclass.aws=gp3 \ 
--set alerts.receivers-(“devâcompany.local”,”devâêesubcompany.com”) N 
--set alerts.receivers[0].host-"smtpdev.subcompany.com" \ 
--values-./path/to/app.dev.yaml \ 
--values-./path/to/alerts.dev.yaml 


РАБОТА СО СПИСКАМИ 


арр: 
databaseurl: postgres://myapp:mypass@postgres : 5432 /БасКепа 
storage: 
persistence: "true" 
storageclass: 
aws: gp2 
gcp: standard = 
persistentVolumeSize: 
aws: 10G 
gcp: 20G = 
alerts: 
from: "info.production(company. local" 
receivers: 


- "театасотрапу. local" 
host: "“smtp.company. local” 
- "support@subcompany.com" 
host: "smtp.subcompany.com" 


СПОСОБЫ ПЕРЕДАЧИ VALUES 


helm upgrade --install -п ci-namespace ci-release path/to/chart \ 
--set global.env=develop \ 
--set app.databaseurl=$POSTGRESQL PASSWORD DEV \ 
--set app.cloud=aws,app.storage.storageclass.aws=gp3 \ 
--set alerts.receivers-(“devâcompany.local”,”devâêesubcompany.com”) N 
--set alerts.receivers[0].host-"smtpdev.subcompany.com*“ N 
--values-./path/to/app.dev.yaml \ 
--values-./path/to/alerts.dev.yaml 


РАБОТА CO СПИСКАМИ 


volumeClaimTemplates: 


{{ 
) {4 


{4 
{4 


(t 
е 


metadata: 
name: app-data 
spec: 
accessModes: [ "ReadWriteOnce" ] 
resources: 
requests: 
lf eq .Values.app.cloud "ам5" }} 


storage: "ii .Values.app.storage.persistenVolumeSize.aws }}" 
else }} 
storage: "ii .Values.app.storage.persistenVolumeSize.gcp }}" 
end }} 
if eq .Values.app.cloud "aws" }} 
storageClassName: {{ .Values.app.storage.storgeclass.aws }} 
else }} 
storageClassName: {{ .Values.app.storage.storgeclass.gcp }} 
end }} 


ныт 


РАБОТА СО СПИСКАМИ 


volumeClaimTemplates: 


it 
{+ 


14 
14 


ü 
ü 


metadata: 
name: app-data 
spec: 
accessModes: [ "ReadWriteOnce" ] 
resources: 
requests: 


if eq .Values.app.cloud "aws" }} 
storage: "ii .Values.app.storage.persistenVolumeSize.aws |)" 
else }} 
storage: "44 .Values.app.storage.persistenVolumeSize.gcp |)" 
end }} 
if eq .Values.app.cloud "aws" }} 
storageClassName: {{ .Values.app.storage.storgeclass.aws }} 
else }} 
storageClassName: {{ .Values.app.storage.storgeclass.gcp }} 
end }} 


РАБОТА СО СПИСКАМИ 


volumeClaimTemplates: 


- metadata: 
name: app-data 
spec: 
accessModes: [ "ReadWriteOnce" ] 
resources: 
requests: 


storage: »- 
{{ first (pluck .Values.app.cloud .Values.app.storage.persistenVolumeSize) | quote }} 
storageClassName: {{ first (pluck .Values.app.cloud .Values.app.storage.storgeclass) }} 


РАБОТА СО СПИСКАМИ 


volumeClaimTemplates: 


- metadata: 
name: app-data 
spec: 
accessModes: [ "ReadWriteOnce" ] 
resources: 


requests: | 
storage: >- 
{{ first (pluck .Values.app.cloud .Values.app.storage.persistenVolumeSize) | quote }} 
storageClassName: {{ first (pluck .Values.app.cloud .Values.app.storage.storgeclass) }} 


РАБОТА СО СПИСКАМИ 


volumeClaimTemplates: 


- metadata: 
name: app-data 
spec: 
accessModes: [ "ReadWriteOnce" ] 
resources: 
requests: 4 
storage: >- 


14 first (pluck .Values.app.cloud .Values.app.storage.persistenVolumeSize) | quote }} 
storageClassName: {{ first (pluck .Values.app.cloud .Values.app.storage.storgeclass) }} 


РАБОТА СО СПИСКАМИ 


volumeClaimTemplates: 


- metadata: 
name: app-data 
spec: 
accessModes: [ "ReadWriteOnce" ] 
resources: 
requests: | 
storage: >- 


{{ first (pluck .Values.app.cloud .Values.app.storage.persistenVolumeSize) | quote }} 
storageClassName: {{ first (pluck .Values.app.cloud .Values.app.storage.storgeclass) }} 


РАБОТА СО СПИСКАМИ 


volumeClaimTemplates: 


- metadata: 
name: app-data 
spec: 
accessModes: [ "ReadWriteOnce" ] 
resources: 


idee | J 
storage% »- 

{{ first (pluck .Values.app.cloud .Values.app.storage.persistenVolumeSize) | quote }} 
storageClassName: {{ first (pluck .Values.app.cloud .Values.app.storage.storgeclass) }} 


СТАНДАРТНЫЕ ЗНАЧЕНИЯ HELM 


Chart: The contents of the Chart.yaml file. Any data in Chart.yaml will be accessible here. 
For example {{ .Chart.Name }}-{{ .Chart.Version }} will print out the mychart-0.1.0. 


Values: Values passed into the template from the values.yaml file and from user-supplied 
files. By default, Values is empty. 


Release: This object describes the release itself. It has several objects inside of it: 
Release.Name: The release name 

Release.Namespace: The namespace to be released into (if the manifest doesn’t override) 
Release.IsUpgrade: This is set to true if the current operation is an upgrade or rollback. 
Release.IsInstall: This is set to true if the current operation is an install. 


Guide: https://helm.sh/docs/chart template guide/builtin objects 


СТАНДАРТНЫЕ ЗНАЧЕНИЯ HELM 


aptVersion: apps/vl 
kind: Deployment 
metadata: 
name: {{ .Chart.Name }}-worker dumm 
spec: 
strategy: 
type: RollingUpdate 
rollingUpdate: 
maxUnavailable: 1 
maxSurge: 1 
selector: 
matchLabels: 
app: {{ .Release.Name }}-worker Gam 
template: 
metadata: 
labels: 
app: {{ .Release.Name }}-worker «amm 


MUSTHAVE УАМЕ-ДЕВЕЛОПЕРА 


1. Кастомизация приложения в разрезе 
окружений 


Переиспользвание кода 


Управление блоками кода по условию 


Модульная разработка 

Трекинг выката 

Установка очерёдности запуска подов 
Отладка 


Релизный цикл 


е O аз т а ы» кы 


=> => 
Расширение функциональности HELM 


YAGNI KISS 


ПЕРЕИСПОЛЬЗОВАНИЕ КОДА 


1 арр: 

2 PYTHONUNBUFFERED: "1" 

3  FLASK APP: "project/__init__.py" 
4 FLASK DEBUG: "1" 

5 APP USERNAME: "throw app" 

6 JSON AS ASCII: "False" 

7 SEND FILE MAX AGE DEFAULT: "0" 

8 THROW DB HOST: "192.168.5.11" 

9  THROW DB NAME: "throw dev" 

19 THROW USER: "throw user" 

11 THROW DB. PASSWORD: "throw test throw test" 


* backend 
° websockets 
* workers 
* cronjobs 


HELM NAMED TEMPLATES 


_* tpl . /mychart 
* [— Chart.yaml 
_*yaml [— charts 
— templates 
|- NOTES. txt 


I— _helpers.tpl 
[— deployment.yaml 
— hpa.yaml 
[— ingress.yaml 
l— service.yaml 
[— serviceaccount.yaml 
L— tests 
L— test-connection.yaml 
L— values.yaml 


{{- define "арр_епу5" }} 


name: 
value 
name: 
value 
name: 


value: 


name. 


маше: 


пате: 


маше: 


пате: 


маше: 


пате: 


маше: 


пате: 


маше: 


пате: 


маше: 


пате: 


маше: 


{{- end 


PYTHONUNBUFFERED 
: {{ .Values.app.PYTHONUNBUFFERED }} 
FLASK_APP 
: {{ .Values.app.FLASK APP }} 
FLASK_DEBUG 

{{ .Values.app.FLASK_DEBUG }} 
APP_USERNAME 

{{ .Values.app.APP_USERNAME }} 
JSON AS ASCII 

{{ .Values.app.JSON AS ASCII }} 
SEND FILE MAX AGE DEFAULT 

{{ .Values.app.SEND FILE MAX AGE DEFAULT }} 
THROW. DB. HOST 

{{ .Values.app.THROW DB HOST }} 
THROW DB. NAME 

{{ .Values.app.THROW DB NAME }} 
THROW USER 

{{ .Values.app.THROW USER }} 
THROW. DB. PASSWORD 

{{ .Values.app.THROW DB PASSWORD }} 
H 


_ envs.tpl 


HELM NAMED TEMPLATES — INCLUDE 


spec: 
contatners: 

- name: myapplication 
image: {{ .Values.global.image name }} 
imagePullPolicy: Always 
env: 

- name: KUBERNETES DEPLOYED 
value: {{ now }} 


mmm) ^ {{- include "app envs" . | indent 12 }} 


ports: 
- name: http 
containerPort: 5000 


NAMED TEMPLATES — LABELS 


14 define "labels" }} 
team: backend 
departament: bigdata 
project: frov-planning 


{{ end }} 


template: 
metadata: 
labels: 
app: 44 .Chart.Name }} 
{{ include "labels" . | indent 8 }} 


NAMED TEMPLATES — ANNOTATIONS 


{{ define "annotations" }} 

annotations: 
pipeline: {{ .Values.global.pipeline | quote }} 
user: 44 .Values.global.pipelinerunner | quote }} 


{{ end }} 


apiVersion: apps/vl 
kind: Deployment 
metadata: 
name: {{ .Chart.Name }} 
{{- include "annotations" . | indent 2}} 


NAMED TEMPLATES — TOLERATIONS 


{{ define "nodeselector" }} 
nodeSelector: 
"mcs-nodepool": "system" 
tolerations: 
- key: dedicated 
operator: Equal 
value: system 


{{ end }} 


podTemplate: 
spec: 
toleratuons: 
{{- include "nodeselector" 


| indent 8 jj 


NAMED TEMPLATES — AFFINITY 


14 define "affinity" }} 
ít if eg .Values.global.ens "production" }} 
affinity: 
podAntiAffinity: 
requiredDuringSchedulingIgnoredDuringExecution: 
- labelSelector: 
matchExpressions: 
- key: app 
operator: In 
values: 
- 44 .Chart.Name Y) 
topologyKey: kubernetes.io/hostname 
лепа р} 
{{ end }} 


template: 
metadata: 
labels: 
app: {{ .Chart.Name }} 
spec: 
{{ include "affinity" . }} 


1 
1 


ПРИМЕР values.yaml 


1 app: 

2 PYTHONUNBUFFERED: "1" 

3  FLASK APP: "project/ init .py" 
4  FLASK DEBUG: "1" 

5 APP USERNAME: "throw, app" 

6 JSON AS ASCII: "False" 

7 SEND FILE. MAX AGE. DEFAULT: "0" 

8 THROW DB HOST: "192.168.5.11" 

9  THROW DB NAME: "throw dev" 

0 THROW USER: "throw user" 

1  THROW DB PASSWORD: "throw test throw test" 


ПРИМЕР DEFINE ДЛЯ env 


{{- define "app_envs" }} 
- name: PYTHONUNBUFFERED 
value: {{ .Values.app.PYTHONUNBUFFERED }} 
- name: FLASK APP 
value: (1 .Values.app.FLASK APP |) 
- name: FLASK DEBUG 
value: (1 .Values.app.FLASK DEBUG )) 
- name: APP USERNAME 
value: (1 .Values.app.APP USERNAME }} 
- name: JSON AS ASCII 
value: 41 .Values.app.JSON AS ASCII }} 
- name: SEND. FILE. МАХ AGE. DEFAULT 
value: 41 .Values.app.SEND FILE MAX AGE DEFAULT }} 
- name: THROW. DB. HOST 
value: (1 .Values.app.THROW DB HOST }} 
- name: THROW DB. NAME 
value: 44 .Values.app.THROW DB NAME }} 
- name: THROW USER 
value: (1 .Values.app.THROW USER }} 
- name: THROW ОВ PASSWORD 
value: {{ .Values.app.THROW DB PASSWORD }} 
епа и 


RANGE #1 — ЗАДАННЫЙ СПИСОК 


{{ range tuple "1" "2" "3" }} 
apiVersion: vl 
kind: PersistentVolume 
metadata: 
name: {{ $.Chart.Name }}-server-{{ . }} 
spec: 
nodeAffinity: 
required: 
nodeSelectorTerms: 
- matchExpressions: 
- key: kubernetes.io/hostname 
operator: In 
values: 
- kube-clickhouse-{{ . }} 
capacity: 
storage: 10Gi 
accessModes: 
- ReadWriteOnce 
persistentVolumeReclaimPolicy: Retain 
storageClassName: clickhouse-server-data 
local: 
path: /mnt/localstorage/clickhouse-server-data 


{{ end }} 


i a #1 — ЗАДАННЫЙ СПИСОК 


{{ range tuple "1" "2" "3" }} 
apiVersion: vl 
kind: PersistentVolume 
metadata: 
name: {{ $.Chart.Name }}-server-{{ . }} 
spec: 
nodeAffinity: 
required: 
nodeSelectorTerms: 
- matchExpressions: 
- key: kubernetes.io/hostname 
operator: In 
values: 
- kube-clickhouse-{{ . }} 
capacity: 
storage: 10Gi 
accessModes: 
- ReadWriteOnce 
persistentVolumeReclaimPolicy: Retain 
storageClassName: clickhouse-server-data 
local: 
path: /mnt/localstorage/clickhouse-server-data 


{{ end }} 


RANGE #1 — ЗАДАННЫЙ СПИСОК 


{{ range tuple "1" "2" "3" }} 
apiVersion: vl 
kind: PersistentVolume 
metadata: | 
пате: 41 $.Chart.Name }}-server-{{ . }} 
spec: 
nodeAffinity: 
required: 
nodeSelectorTerms: 
- matchExpressions: 
- key: kubernetes.io/hostname 
operator: In 
values: 4 
- kube-clickhouse-{{ . || 
Capacity: 
storage: 10Gi 
accessModes: 
- ReadWriteOnce 
persistentVolumeReclaimPolicy: Retain 
storageClassName: clickhouse-server-data 
local: 
path: /mnt/localstorage/clickhouse-server-data 


{{ end }} 


RANGE #1 — ЗАДАННЫЙ СПИСОК 


{{ range tuple "1" "2" "3" }} 
apiVersion: vl 
kind: PersilWtentVolume 
metadata: 
name: {{ $.Chart.Name }}-server-{{ . )) 
spec: 
nodeAffinity: 
required: 
nodeSelectorTerms: 
- matchExpressions: 
- key: kubernetes.io/hostname 
operator: In 
values: 
- kube-clickhouse-{{ . }} 
capacity: 
storage: 10Gi 
accessModes: 
- ReadWriteOnce 
persistentVolumeReclaimPolicy: Retain 
storageClassName: clickhouse-server-data 
local: 
path: /mnt/localstorage/clickhouse-server-data 


{{ end }} 


RANGE #2 – values.yaml 


1 cronjobs: 

2 # Подготовительные мероприятия к старту нового сезона (за час до старта) 
3 # 23:00 MSK (20:00 UTC) 

4 prepare-start-season: 

5 schedule: "0 20 х х x" 

6 command: '["bash"]' 

7 args: '["-c", "/app/src/manage.py start period"]' 

8 # Мероприятия no закрытию периода 

9 # 12:00 MSK (09:00 UTC) 

10 complete-period: 


11 schedule: "99 хх x" 
12 command: '["bash"]' 
13 args: '["-c", "/app/src/manage.py complete period"]' 


14 # Отправка наград 

15 # 03:05 MSK (00:05 UTC) 

16 send-pending-rewards: 

17 schedule: "5 0 хх x" 

18 command: '["bash"]' 

19 args: '["-c", "/app/src/manage.py send pending rewards --ignore-previous" |! 


RANGE #2 — template 


{{- range $name, $params := .Values.cronjobs }} 
apiVersion: batch/v2alphal 
kind: CronJob 
metadata: 
name: {{ $.Chart.Name }}-{{ $name }} 
spec: 
schedule: {{ $params.schedule | quote}} 
concurrencyPolicy: Forbid 


jobTempLate: 
spec: 
template: 
spec: 
containers: 
- name: job 


command: {{ $params.command }} 
args: {{ $params.args }} 
image: {{ $.Values.global.image_name }} 
env: 
{{ include "common_envs" $ | indent 14 }} 
restartPolicy: OnFailure 


{{- end }} 


RANGE "f — тае 


{{- range $name, $params := .Values.cronjobs }} 
apiVersion: batch/v2alphal 
kind: CronJob 
metadata: 
name: {{ $.Chart.Name }}-{{ $name }} 
spec: 
schedule: {{ $params.schedule | quote}} 
concurrencyPolicy: Forbid 


jobTempLate: 
spec: 
template: 
spec: 
containers: 
- name: job 


command: {{ $params.command }} 
args: {{ $params.args }} 
image: {{ $.Values.global.image name }} 
env: 
{{ include "common envs" $ | indent 14 }} 
restartPolicy: OnFailure 


{{- end }} 


RANGE #2 — template 


{{- range $name, $params := .Values.cronjobs }} 
apiVersion: batch/v2alphal 
kind: CronJob 
metadata: 
name: {{ $.Chart.Name }}-{{ $name }} 
spec: 
schedule: {{ $params.schedule | quote}} 
concurrencyPolicy: Forbid 


jobTempLate: 
spec: 
template: 
spec: 
containers: 
- name: job 


command: {{ $params.command }} 
args: {{ $params.args }} 
image: {{ $.Values.global.image_name }} 
env: 
{{ include "common_envs" $ | indent 14 }} 
restartPolicy: OnFailure 


{{- end }} 


RANGE #2 — template 


{{- range $name, $params := .Values.cronjobs }} 
apiVersion: batch/v2alphal 
kind: CronJob 
metadata: 
name: {{ $.Chart.Name }}-{{ $name }} 
spec: 
schedule: {{ $params.schedule | quote}} p 


concurrencyPolicy: Forbid 


jobTemplate: 
spec: 
template: 
spec: 
containers: 
- name: job 


command: {{ $params.command }} == 
args: {{ $params.args }} 
image: 44 $.Values.global.image name }} 
env: 
{{ include "common envs" $ | indent 14 }} 
restartPolicy: OnFailure 


{{- end }} 


RANGE #2 — template 


{{- range $name, $params := .Values.cronjobs }} 
apiVersion: batch/v2alphal 
kind: CronJob 
metadata: 
name: {{ $.Chart.Name }}-{{ $name }} 
spec: 
schedule: {{ $params.schedule | quote}} 
concurrencyPolicy: Forbid 


jobTempLate: 
spec: 
template: 
spec: 
containers: 
- name: job 


command: {{ $params.command }} 
args: {{ $params.args }} 
image: 44 $.Vglues.global.image name }} 
env: 
{{ include "common envs" $ | indent 14 }} 
restartPolicy: OnFailure 


{{- end }} 


RANGE #2 — template 


{{- range $name, $params := .Values.cronjobs }} [== 
apiVersion: batch/v2alphal 
kind: CronJob 
metadata: 
name: {{ $.Chart.Name }}-{{ $name }} 
spec: 
schedule: {{ $params.schedule | quote}} 
concurrencyPolicy: Forbid 


jobTempLate: 
spec: 
template: 
spec: 
containers: 
- name: job 


command: {{ $params.command }} 
args: {{ $params.args }} 
image: {{ $.Values.global.image_name }} 
env: 
{{ include "common_envs" $ | indent 14 }} 
restartPolicy: OnFailure 


{{- end }} 


RANGE #2 – template 


{{- range $name, $params := .Values.cronjobs }} qma 


apiVersion: batch/v2alphal 
kind: CronJob 
metadata: 
name: {{ $.Chart.Name }}-{{ $name }} 
spec: 
schedule: {{ $params.schedule | quote}} 
concurrencyPolicy: Forbid 


jobTempLate: 
spec: 
template: 
spec: 
containers: 
- name: job 


command: {{ $params.command }} 
args: {{ $params.args }} 
image: 44 $.Values.global.image name }} 
env: 
{{ include "common envs" $ | indent 14 }} 
restartPolicy: OnFailure 


{{- end }} 


ПРИМЕР DEFINE ДЛЯ env 


{{- define "app_envs" }} 
- name: PYTHONUNBUFFERED 
value: {{ .Values.app.PYTHONUNBUFFERED }} 
- name: FLASK APP 
value: (1 .Values.app.FLASK APP |) 
- name: FLASK DEBUG 
value: (1 .Values.app.FLASK DEBUG )) 
- name: APP USERNAME 
value: (1 .Values.app.APP USERNAME }} 
- name: JSON AS ASCII 
value: 41 .Values.app.JSON AS ASCII }} 
- name: SEND. FILE. МАХ AGE. DEFAULT 
value: 41 .Values.app.SEND FILE MAX AGE DEFAULT }} 
- name: THROW. DB. HOST 
value: (1 .Values.app.THROW DB HOST }} 
- name: THROW DB. NAME 
value: 44 .Values.app.THROW DB NAME }} 
- name: THROW USER 
value: (1 .Values.app.THROW USER }} 
- name: THROW ОВ PASSWORD 
value: {{ .Values.app.THROW DB PASSWORD }} 
епа и 


ПРИМЕР RANGE ДЛЯ env 


{{- define “app_envs" }} 


{{- range $name, $value :- .Values.app }} 
- name: {{ $name }} 

value: {{ $value }} 
tieng. 


{{- end Б 


ПРИМЕР RANGE ДЛЯ env 


{{- define "app_envs" }} 


{{- range $name, $value :- .Values.app }} 
- name: {{ $name }} 

value: {{ $value }} 
end 


- name: GIT COMMIT 
value: "{{ .Values.global.git_commit }}" 
- name: GIT TAG 
value: "{{ .Values.global.git_tag }}" 


{{- end }} 


ОДИН values.yaml ДЛЯ ВСЕХ ОКРУЖЕНИЙ 


1 арр: 

2 PYTHONUNBUFFERED: "1" 

З FLASK APP: "project/__init__.py" 
4  FLASK DEBUG: "1" 

5.  APP_USERNAME: "throw app" 

6. JSON AS ASCII: "False" 

7 | SEND FILE MAX AGE DEFAULT: "0" 

8 THROW DB. HOST: "192.168.5.11" 

9  THROW DB NAME: "throw dev" 

19 THROW USER: "throw user" 

11 THROW DB. PASSWORD: "throw test throw test" 


ОДИН values.yaml ДЛЯ ВСЕХ ОКРУЖЕНИЙ 


арр: 
PYTHONUNBUFFERED: 
default: "1" 
FLASK APP: 


_default: "project/ init .py 
FLASK DEBUG: 

-dgefault: "1^ 
APP USERNAME: 

_default: "throw app" 
JSON AS ASCII: 

default: "False" 
THROW. DB. HOST: 

default: -192.168.5.21' 

stage: "192.168.4.9" 

prod: "192.168.4.91" 
THROW. DB. NAME: 

default: "throw dev" 

test: "throw" 

stage: "throw preprod" 

prod: "pproduction" 


ОДИН values.yaml ДЛЯ ВСЕХ ОКРУЖЕНИЙ 


{{- define "app envs" )) 


{{- range $name, $map := .Values.app }} 
{{- $value := pluck $.Values.global.env $map | first | default $map._default }} 
- name: {{ $name }} 
value: {{ $value | quote}} 
{{- end }} 


{{- end }} 


ОДИН values.yaml ДЛЯ ВСЕХ ОКРУЖЕНИЙ 


{{- define “app_envs" }} 


{{- range $name, $map := .Values.app }} | 
{{- $value := pluck $.Values.global.env $map | first | default $map._default }} 
- name: {{ $name }} 
value: {{ $value | quote}} 
епа 


{{- епа }} 


HELM TIPS AND TRICKS 


Chart best practices 
https://helm.sh/docs/chart best practices 

Chart tips and tricks 
https://helm.sh/docs/howto/charts tips and tricks 


Helm template function list 
https://helm.sh/docs/chart template guide/function list 
Sprig Function Documentation 
http://masterminds.github.io/spri 


HELM TIPS AND TRICKS 


kind: Deployment 
spec: 
template: 
metadata: 
annotations: 4 
checksum/config: >- 
(1 include (print $.Template.BasePath "/configmap.yaml") . | sha256sum }} 


MUSTHAVE УАМЕ-ДЕВЕЛОПЕРА 


1. Кастомизация приложения в разрезе 
окружений 


Переиспользвание кода 


Управление блоками кода по условию 


Модульная разработка 

Трекинг выката 

Установка очерёдности запуска подов 
Отладка 


Релизный цикл 


> аз п ae Мм 


=> => 
Расширение функциональности HELM 


YAGNI KISS 


УПРАВЛЕНИЕ БЛОКАМИ КОДА 


{{- if and (ne .Values.global.env 'production”) (ne .Values.global.env “loadtest") Y) 


apiVersion: apps/vl 
kind: StatefulSet 


metadata: 

name: {{ .Chart.Name ))-postgresgl 
spec: 

serviceName: postgres 

selector: 

matchLabels: 
service: postgresgl 

| 


{{- епа }} 


УПРАВЛЕНИЕ БЛОКАМИ КОДА 


{{ if .Values.app.storage.persistence }} 
volumeMounts: 
- mountPath: /data 
name: data 
volumeClaimTemplates: 
- metadata: 
name: app-data 
spec: 
accessModes: [ "ReadWriteOnce" ] 
resources: 
requests: 
storage: 45 first (pluck .Values.app.cloud .Values.app.storage.persistenVolumeSize) | quote jj 
storageClassName: {{ first (pluck .Values.app.cloud .Values.app.storage.storgeclass) }} 
{{ else }} 
volumeMounts: 
- mountPath: /data 
emptyDir: {} 
LU end I+ 


УПРАВЛЕНИЕ БЛОКАМИ КОДА 


spec: 
containers: 
- name: myapplication 
{{- if eq .Values.global.env 'production” }} 
resources: 
1 requests: 
cpu: 3500m 
{{- else if hasPrefix “review” .Values.global.env || 
resources: 
requests: 
cpu: 500m 
ЕЕ a 
resources: 
limits: 
cpu: 1500m 
{{- end }} 


ФУНКЦИЯ toYaml — values.yaml 


resources: 
prod: 
Limits: 
cous 2 
memory: 461 
requests: 
EDnu c1 
memory: 2Gi 
stage: 
limits: 
cpu: 500m 
memory: 161 
requests: 
cpu: 250m 
memory: 500Mi 


ФУНКЦИЯ toYaml – templating 


spec: 
contatners: 
- name: myapp 
resources: 


14 toYaml .Values.resources.prod | indent 12 )) 


t 


ФУНКЦИЯ toYaml — templating 


Spec: 
contatners: 
- name: myapp 
resources: 


{{ toYaml (pluck .Values.global.env .Values.resources | first) | indent 12 }} 


t 


MUSTHAVE УАМЕ-ДЕВЕЛОПЕРА 


1. Кастомизация приложения в разрезе 
окружений 


Переиспользвание кода 


Управление блоками кода по условию 


Модульная разработка 

Трекинг выката 

Установка очерёдности запуска подов 
Отладка 


Релизный цикл 


ее аз LE Ж. I 


=> => 
Расширение функциональности HELM 


YAGNI KISS 


“TAK ИСТОРИЧЕСКИ СЛОЖИЛОСЬ” 


. helm 


— 
— 


— 


Chart.yaml 

templates 

20-postgres.yaml 
40-celery-beat.yaml 
41-store-messages.yaml 
42-reply-to-clients.yaml 
50-listening-workers.yaml 
60-app.yaml 

61-api.yaml 
65-metrics.yaml 
70-rabbitmq.yamLl 
90-ingress.yaml 
99-postgres-backup.yaml 
.envs.tpl 
cronjob-archive-messages.yaml 
job-archive-messages.yaml 
job-custom-tasks.yaml 
job-psql-flush-db.yaml 
job-migrate.yaml 

L— job-tests.yaml 
values.yaml 


Pee ATA AFU 


HELM МОДУЛЬНАЯ РАЗРАБОТКА 


./mychart 
[— Chart.yaml 
I— charts 
— templates 
— NOTES. txt 
I— _helpers.tpl 
[— deployment.yaml 
— hpa.yaml 
[— ingress.yaml 
l— service.yaml 
[— serviceaccount.yaml 
L— tests 
L— test-connection.yaml 
L— values.yaml 


HELM МОДУЛЬНАЯ РАЗРАБОТКА 


./mychart 

[— Chart.yaml 

— charts 

I— backend 
ingress-tls 
job-load-db-dump 
job-mc-create-bucket 
job-migrations 
minio 

postgres 
postgres-backup 
rabbitmq 
emplates 

L— envs.tpl 
values.yaml 


ІІТІТІТІ 


mE 


HELM SUBCHARTS Chart.yaml 


apiVersion: v2 
name: throw-backend 
version: 0.0.1 


dependencies: 
- name: minio 
version: 6.2.11 
repository: https://charts.bitnami.com/bitnami 
condition: minio.enabled, global.contractTests.enabled 
- name: backend 
version: 0.1 
repository: file://path/to/backend/chart 
condition: backend.enabled 
- name: postgres 
version: 13.0.1 
repository: "@stable" 
condition: postgres.enabled 
- name: rabbitmq 
version: 0.1 
condition: rabbitmq.enabled, global.contractTests.enabled 


HELM SUBCHARTS Chart.yaml 


apiVersion: v2 
name: throw-backend 
version: 0.0.1 


dependencies: 
- name: minio 
version: 6.2.11 
repository: https://charts.bitnami.com/bitnami 
condition: minio.enabled, global.contractTests.enabled 
- name: backend 
version: 0.1 
repository: file://path/to/backend/chart 
condition: backend.enabled 
- name: postgres 
version: 13.0.1 
repository: "Qstable" 
condition: postgres.enabled 
- name: rabbitmq 
version: 0.1 
condition: rabbitmq.enabled, global.contractTests.enabled 


HELM SUBCHARTS Chart.yaml 


apiVersion: v2 
name: throw-backend 
version: 0.0.1 


dependencies: 
- name: minio 
version: 6.2.11 
repository: https://charts.bitnami.com/bitnami 
condition: minio.enabled, global.contractTests.enabled 
- name: backend 
version: 0.1 
repository: file://path/to/backend/chart 
condition: backend.enabled 
- name: postgres 
version: 13.0.1 
repository: "@stable" 
condition: postgres.enabled 
- name: rabbitmq 
version: 0.1 
condition: rabbitmq.enabled, global.contractTests.enabled 


HELM SUBCHARTS Chart.yaml 


apiVersion: v2 
name: throw-backend 
version: 0.0.1 


dependencies: 
- name: minio 
version: 6.2.11 
repository: https://charts.bitnami.com/bitnami 
condition: minio.enabled, global.contractTests.enabled 
- name: backend 
version: 0.1 
repository: file://path/to/backend/chart 
condition: backend.enabled 
- name: postgres 
version: 13.0.1 
repository: "@stable" 
condition: postgres.enabled 
- name: rabbitmq 
version: 0.1 
condition: rabbitmq.enabled, global.contractTests.enabled 


HELM SUBCHARTS Chart.yaml 


apiVersion: v2 
name: throw-backend 
version: 0.0.1 


dependencies: 
- name: minio 
version: 6.2.11 
repository: https://charts.bitnami.com/bitnami 
condition: minio.enabled, global.contractTests.enabled 
- name: backend 
version: 0.1 
repository: file://path/to/backend/chart 
condition: backend.enabled 
- name: postgres 
version: 13.0.1 
repository: "@stable" 
condition: postgres.enabled 
- name: rabbitmq 
version: 0.1 
condition: rabbitmq.enabled, global.contractTests.enabled 


РАБОТА С ВНЕШНИМИ РЕПОЗИТОРИЯМИ 


apiVersion: v2 
name: throw-backend 
version: 0.0.1 


dependencies: 
- name: minio 
version: 6.2.11 
repository: https://charts.bitnami.com/bitnami 
condition: minio.enabled, global.contractTests.enabled 
- name: backend 
version: 0.1 
repository: file://path/to/backend/chart 
condition: backend.enabled 
- name: postgres 
version: 13.0.1 
repository: "@stable" 
condition: postgres.enabled 


helm dependency update 


УСЛОВИЯ ВКЛЮЧЕНИЯ ЧАРТА 


apiVersion: v2 
name: throw-backend 
version: 0.0.1 


dependencies: 
- name: minio 
version: 0.1 
condition: minio.enabled, global.contractTests.enabled 
tags: 
- environment 
- contracttests 
- name: backend 
version: 0.1 
condition: backend.enabled, global.contractTests.enabled 
tags: 
- environment 
- contracttests 
- name: job-mc-create-bucket 
version: 0.1 
condition: job-mc-create-bucket.enabled 
- name: job-migrations 
version: 0.1 
tags: 
- migrations 


УСЛОВИЯ ВКЛЮЧЕНИЯ ЧАРТА 


apiVersion: v2 
name: throw-backend 
version: 0.0.1 


dependencies: 
- name: minio 
version: 0.1 
condition: minio.enabled, global.contractTests.enabled 
tags: 
- environment 
- contracttests 
- name: backend 
version: 0.1 
condition: backend.enabled, global.contractTests.enabled 
tags: 
- environment 
- contracttests 
- name: job-mc-create-bucket 
version: 0.1 
condition: job-mc-create-bucket.enabled 
- name: job-migrations 
version: 0.1 
tags: 
- migrations 


УСЛОВИЯ ВКЛЮЧЕНИЯ ЧАРТА 


apiVersion: v2 
name: throw-backend 
version: 0.0.1 


dependencies: 
- name: minio 
version: 0.1 
condition: minio.enabled, global.contractTests.enabled 
tags: 
- environment 
- contracttests 
- name: backend 
version: 0.1 
condition: backend.enabled, global.contractTests.enabled 
tags: 
- environment 
- contracttests 
- name: job-mc-create-bucket 
version: 0.1 
condition: job-mc-create-bucket.enabled 
- name: job-migrations 
version: 0.1 
tags: 
- migrations 


CONDITIONS и TAGS - values.yaml 


# conditions 
minio: 
enabled: true 
postgresql: 
enabled: true 
backend: 
enabled: true 
rabbitmq: 
enabled: true 
Job-load-db-dump: 
enabled: false 
# tags 
tags: 
environment: true 
migrations: false 


CONDITIONS и TAGS — вызов HELM 


helm upgrade --install -n $(CI NAMESPACE} ${RELEASENAME} .infra/helm N 
--set "backend.enabled-true" \ 
--set "minio.enabled-true" \ 
--set "postgres.enabled-true" \ 
--set "rabbitmq.enabled-true" ÓN 


--set "tags.migrations-false" 


VALUES И SUBCHARTS 


global: 
system: 
storageclass: "hdd" 
tests: "no" 
app: 
DB URL: postgres://myapp:mypass@postgres :5432/backend 
RMQ URL: amgp://rabbitmg:mypasserabbitmg 


postgres: 
POSTGRES USER: "backend" 
POSTGRES PASSWORD: "mypassword" 


minio: 
storage: "261" 
MINIO KEY: '123123123' 


rabbitmg: 
rmg: 
pvc: 
size: '2Gi' 


VALUES И SUBCHARTS 
=z global: 


system: 

Storageclass: "hdd" 
tests: "no" 
app: 


DB URL: postgres://myapp:mypass@postgres :5432/backend 
RMQ URL: amgp://rabbitmg:mypasserabbitmg 


postgres: 
POSTGRES USER: "backend" 
POSTGRES PASSWORD: "mypassword" 


minio: 
storage: "261" 
MINIO KEY: '123123123' 


rabbitmg: 
rmg: 
pvc: 
size: '2Gi' 


VALUES И SUBCHARTS 


global: 
system: 
storageclass: "hdd" 
tests: "no" 
app: 


DB URL: postgres://myapp:mypass@postgres :5432/backend 
RMQ URL: amgp://rabbitmg:mypasserabbitmg 


= postgres: 


POSTGRES_USER: "backend" 
POSTGRES PASSWORD: "mypassword" 


7 minio: 
storage: "261" 
MINIO KEY: '123123123' 


EN) 17 rabbitmg: 


rmq: 
pvc: 
size: '2Gt' 


NAMED TEMPLATES И SUBCHARTS 


|-- postgres-backup 
L— rabbitmq 


templates 
L—  envs.tpl 
values.yaml 


./mychart apiVersion: v2 
[— Chart.yaml name: mylibchart 
[— charts description: A Helm chart for Kubernetes 
| F— backend шшщ) 4 type: library 
| FF ingress-tls version: 0.1.0 
| F— job-load-db-dump appVersion: "1.16.0" 
| FF job-mc-create-bucket 
| | job-migrations 
| FF minio 
| FF postgres 
| 
| 
==} 
- 


_# tpl *.yaml Library charts 


NAMED TEMPLATES — AFFINITY 


{{ define "affinity" }} 
ít if eq .Values.global.ens “production” }} 
affinity: 
podAnttAffinity: 
requiredDuringSchedulingIgnoredDuringExecution: 
- labelSelector: 
matchExpressions: 
- key: app 
operator: In 
values: 
- {{ .Chart.Name }} 
topologyKey: kubernetes.io/hostname 


{{ end Б 
{{ end }} 


NAMED TEMPLATES — AFFINITY 


{{ define "affinity" }} 
{{ if eq .Values.global.ens "production" }} 
affinity: 
podAntiAffinity: 
requiredDuringSchedulingIgnoredDuringExecution: 
- labelSelector: 
matchExpressions: 
- key: app 
operator: In 
values: 
шшш) - (( .Chart.Name }} 
topologyKey: kubernetes.io/hostname 
{{ end }} 
{{ end }} 


РАБОТА С ВНЕШНИМИ РЕПОЗИТОРИЯМИ 


= О x 
[6] Artifact Hub х + 


< > С ё artifacthubio/packages/search?page- 1&kind-0 & ж + © : 


à RFTA @ 
O аманы нение sionin @- 


FILTERS @ Reset 
ú kube-prometheus-stack 
| | Official © 
ORG: Prometheus REPO: prometheus-commu... 
|_| Verified publishers © VERSION: 13.13.1 APP VERSION: 0.45.0 
Е kube-prometheus-stack collects Kubernetes manifests, Grafana dashboards... 
Updated 21 hours ago * 73 = Helm chart 
Helm charts (2397) 
в |o ЕЯ Verified Publisher 


|. | Krew kubectl plu... (141) 


РАБОТА С ВНЕШНИМИ РЕПОЗИТОРИЯМИ 


(6) kube-prometheus-stack 13.13.1. X + i H x 
< C 8 artifacthub.io/packages/helm/prometheus-community/kube-prometheus-stack G) *ŵ » [0] š 
e Helm 3+ application ~ 


KUBERNETES VERSION 


Get Repo Info >=1.16.0-0 


LINKS 


manas helm repo add prometheus-community https://prometheus-commun: k reece 


helm repo update 
$ Chart Source 


$ Upstream Project 


See helm repo for command documentation. MAINTAINERS 
& Xtigyro 

Install Chart @ scottrigby 

4% gianrubio 

# Helm & gkarthiks 


$ helm install [RELEASE NAME] prometheus-community/kube-prome 
DEPENDENCIES 


Ё CP kube-state-metrics @ 2... 


C9 prometheus-node-expor... 


РАБОТА С ВНЕШНИМИ РЕПОЗИТОРИЯМИ 


:»> helm repo add prometheus-community https://prometheus-community.github.io/helm-charts 
has been added to your repositories 

:»> helm repo update 

Hang tight while we grab the latest from your chart repositories... 

.. successfully got an update from the "prometheus-community" chart repository 

Update Complete. Happy Helming!& 

:~> helm search repo prometheus-operator 

NAME CHART VERSION APP VERSION DESCRIPTION 
prometheus-community/prometheus-operator О 5:2 0.38.1 DEPRECATED - This 
:»> helm fetch prometheus-community/prometheus-operator 

:»> tar -xvf prometheus-operator-9.3.2.tgz 

prometheus-operator/Chart.yaml 

prometheus-operator/values.yaml 

prometheus-operator/templates/NOTES.txt 

prometheus-operator/templates/ helpers.tpl 
prometheus-operator/templates/alertmanager/alertmanager.yaml 
prometheus-operator/templates/alertmanager/ingress.yaml 
prometheus-operator/templates/alertmanager/ingressperreplica.yaml 


РАБОТА С ВНЕШНИМИ РЕПОЗИТОРИЯМИ 


:»> helm repo add prometheus-community https://prometheus-community.github.io/helm-charts 
has been added to your repositories 

:»> helm repo update 

Hang tight while we grab the latest from your chart repositories... 

.. successfully got an update from the "prometheus-community" chart repository 

Update Complete. Happy Helming!& 

:~> helm search repo prometheus-operator 

NAME CHART VERSION APP VERSION DESCRIPTION 
prometheus-community/prometheus-operator О 5:2 0.38.1 DEPRECATED - This 
:»> helm fetch prometheus-community/prometheus-operator 

:~> tar -xvf prometheus-operator-9.3.2.tgz 

prometheus-operator/Chart.yaml 

prometheus-operator/values.yaml 

prometheus-operator/templates/NOTES.txt 

prometheus-operator/templates/ helpers.tpl 
prometheus-operator/templates/alertmanager/alertmanager.yaml 
prometheus-operator/templates/alertmanager/ingress.yaml 
prometheus-operator/templates/alertmanager/ingressperreplica.yaml 


РАБОТА С ВНЕШНИМИ РЕПОЗИТОРИЯМИ 


:»> helm repo add prometheus-community https://prometheus-community.github.io/helm-charts 
has been added to your repositories 
:»> helm repo update 
Hang tight while we grab the latest from your chart repositories... 
.. successfully got an update from the "prometheus-community" chart repository 
Update Complete. Happy Helming!& 

mm»: helm search repo prometheus-operator 
NAME CHART VERSION APP VERSION DESCRIPTION 
prometheus-community/prometheus-operator О 5:2 0.38.1 DEPRECATED - This 
:»> helm fetch prometheus-community/prometheus-operator 
:~> tar -xvf prometheus-operator-9.3.2.tgz 
prometheus-operator/Chart.yaml 
prometheus-operator/values.yaml 
prometheus-operator/templates/NOTES.txt 
prometheus-operator/templates/ helpers.tpl 
prometheus-operator/templates/alertmanager/alertmanager.yaml 
prometheus-operator/templates/alertmanager/ingress.yaml 
prometheus-operator/templates/alertmanager/ingressperreplica.yaml 


РАБОТА С ВНЕШНИМИ РЕПОЗИТОРИЯМИ 


:»> helm repo add prometheus-community https://prometheus-community.github.io/helm-charts 

has been added to your repositories 

:»> helm repo update 

Hang tight while we grab the latest from your chart repositories... 

.. successfully got an update from the "prometheus-community" chart repository 

Update Complete. Happy Helming!& 

:~> helm search repo prometheus-operator 

NAME CHART VERSION APP VERSION DESCRIPTION 

prometheus-community/prometheus-operator О 5:2 0.38.1 DEPRECATED - This 
> ^ helm fetch prometheus-community/prometheus-operator 

:~> tar -xvf prometheus-operator-9.3.2.tgz 

prometheus-operator/Chart.yaml 

prometheus-operator/values.yaml 

prometheus-operator/templates/NOTES.txt 

prometheus-operator/templates/ helpers.tpl 

prometheus-operator/templates/alertmanager/alertmanager.yaml 

prometheus-operator/templates/alertmanager/ingress.yaml 

prometheus-operator/templates/alertmanager/ingressperreplica.yaml 


MUSTHAVE УАМЕ-ДЕВЕЛОПЕРА 


1. Кастомизация приложения в разрезе 
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Управление блоками кода по условию 


Модульная разработка 

Трекинг выката 

Установка очерёдности запуска подов 
Отладка 


Релизный цикл 


ез аз m ы 


=> => 
Расширение функциональности HELM 


YAGNI KISS 


ТРЕКИНГ ВЫКАТА 


helm upgrade --install -n ci-namespace ci-release path/to/chart \ 


wait.go: 
wait.go: 
wait.go: 
:244: [debug] Deployment is 
wait.go: 
upgrade. 


wait.go 


Release 


--wait 
--timeout 600s 
- -debug 


53: [debug] beginning wait 
244: [debug] Deployment is 
244: [debug] Deployment is 


244: [debug] Deployment is 


\ 
\ 


for 6 resources with timeout of 10m0s 


not 
not 
not 
not 


ready: app-test/app. 0 out of 1 expected pods 
ready: app-test/app. 0 out of 1 expected pods 
ready: app-test/app. 0 out of 1 expected pods 
ready: app-test/app. 0 out of 1 expected pods 


go:138: [debug] updating status for upgraded release for app-test 
"app-test" has been upgraded. Happy Helming! 
NAME: app-test 


are 
are 
are 
are 


ready 
ready 
ready 
ready 


KUBERNETES DEPLOYMENT STRATEGIES 


К -п myapp get deployments.apps 


NAME READY | UP-TO-DATE AVAILABLE AGE 
myapp LP 1 1 151d 
spec: Spec: 
Strategy: Strategy: 

type: RollingUpdate type: RollingUpdate 

rollingUpdate: rollingUpdate: 
maxUnavailable: 0 maxUnavailable: 1 
maxSurge: 1 maxSurge: 1 


€» 


OTKAT BbIKATA 


° Ручной откат 


helm rollback -п ci-namespace ci-release [REVISION] 


° Автоматический откат 
helm upgrade --install -n ci-namespace ci-release path/to/chart \ 


--atomic 


MUSTHAVE УАМЕ-ДЕВЕЛОПЕРА 
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O юм > т а ы Мм 


=> => 
Расширение функциональности HELM 


YAGNI KISS 


HELM LIFECYCLE HOOKS 


pre-install Executes after templates are rendered, but before any 
resources are created in Kubernetes 

post-install Executes after all resources are loaded into 
Kubernetes 

pre-delete Executes on a deletion request before any resources 
are deleted from Kubernetes 

post-delete Executes on a deletion request after all of the 


release's resources have been deleted 


pre-upgrade Executes on an upgrade request after templates are 
rendered, but before any resources are updated 


post-upgrade Executes on an upgrade request after all resources 
have been upgraded 


pre-rollback Executes ona rollback request after templates are 
rendered, but before any resources are rolled back 


post-rollback Executes оп а rollback request after all resources 
have been modified 


test Executes when the Helm test subcommand is invoked ( 
view test docs) 


Guide: https://helm.sh/docs/topics/charts hooks/ 


HELM LIFECYCLE HOOKS 


apiVersion: batch/vl 
kind: Job 
metadata: 
name: migration 
annotations: 
"helm.sh/hook": post-install,pre-upgrade 
"helm.sh/hook-weight": "5" 
"helm.sh/hook-delete-policy": hook-succeeded 
spec: 
activeDeadlineSeconds: 600 
template: 
spec: 
containers: 
- name: migration 
command: ['/bin/sh', '-c', 'python manage.py migrate'] 
image: {{ .Values.global.image name }} 
env: 
{{- include "app envs" . | indent 12 }} 
name: migration 
restartPolicy: Never 
imagePullSecrets: 
- name: registrysecret 
backoffLimit: 2 


HELM LIFECYCLE HOOKS 


apiVersion: batch/v1 
kind: Job 
metadata: 
name: migration 
annotations: 
"helm.sh/hook": post-install,pre-upgrade 
"helm.sh/hook-weight": "5" = 
"helm.sh/hook-delete-policy": hook-succeeded 
spec: 
activeDeadlineSeconds: 600 
template: 
spec: 
containers: 
- name: migration 
command: ['/bin/sh', '-c', 'python manage.py migrate' ] 
image: {{ .Values.global.image_name }} 
env: 
{{- include "app envs" . | indent 12 }} 
name: migration 
restartPolicy: Never 
imagePullSecrets: 
- name: registrysecret 
backoffLimit: 2 


MUSTHAVE УАМЕ-ДЕВЕЛОПЕРА 
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> eee п аш De 


=> => 
Расширение функциональности HELM 


YAGNI KISS 


HELM CHART DEBUG - LINTER 


:~> helm lint ./mychart --set global.env-develop 
==> [inting ./mychart 
[INFO] Chart.yaml: icon is recommended 


1 chart(s) linted, 0 chart(s) failed 


HELM CHART DEBUG — TEMPLATE 


:~> helm template ./mychart --set global.env-develop 
# Source: mychart/templates/20-mychart.yaml 
apiVersion: vl 

kind: Service 


metadata: 
name: mychart 
labels: 
app: mychart 
spec: 
clusterIP: None 
DO TS: 
- name: http 
port: 8080 
selector: 


app: mychart 


--debug 


HELM CHART DEBUG — DRY RUN 


helm install / helm upgrade --dry-run 


:»> helm upgrade --dry-run -n mychart-test myapp-test ./mychart --set global.env-develop 
Release "mapp-test" has been upgraded. Happy Helming! 
NAME: mapp-test 

LAST DEPLOYED: Mon May 13 15:09:57 2021 

NAMESPACE: mychart-test 

STATUS: pending-upgrade 

REVISION: 4 

TEST SUITE: None 

HOOKS: 

MANIFEST: 

# Source: mychart/templates/20-mychart.yaml 
apiVersion: vl 

kind: Service 


ДОКУМЕНТАЦИЯ 


./mychart 
[— Chart.yaml 
[— charts 
— templates 
— NOTES. txt 
[— _helpers.tpl 
[— deployment.yaml 
|-- hpa.yaml 
[— ingress.yaml 
I— service.yaml 
I— serviceaccount.yaml 
L— tests 
L— test-connection.yaml 
L— values.yaml 


NOTES.txt — информация о релизе 


Деплой релиза {{ .Release.Name }} 

в окружение {{ .Values.global.env }} прошел! ° NOTES.txt 
Приложение установлено в namespace: 

- {{ .Release.Namespace || 

Вэб-интерфейс доступен по адресу: 

http://{{ .Values.global.ci_url }} 


Деплой релиза myapp-test 

в окружение test прошел! 

Приложение установлено в namespace: 

- mychart-test 

Вэб-интерфейс доступен по адресу: 
http://mychart-test.kube.company. local 


° ВЫВОД 


КОМЕНТАРИИ 


YAML comments: 


# This is a comment 


type: sprocket 


Template Comments: 


i y da 
This is a comment. 


ый 
type: frobnitz 


152 


HELM ПРОСМОТР/УДАЛЕНИЕ РЕЛИЗОВ 


e helm list -n namespace 
e helm get manifest -n namespace release-name 


:~> helm -n mychart-test list -a 
NAME NAMESPACE REVISION UPDATED STATUS 
myapp-test mychart-test 3 2021-05-13 12:44:47.467142093 +0000 UTC deployed 
:»> helm -n keycloak-test get manifest keycloak-test 
# Source: mychart/templates/20-mychart.yaml 
apiVersion: vl 
kind: Service 
metadata: 

name: mychart 

labels: 

app: mychart 


e helm -n namespace delete release-name 
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ко аз п а ы De 


=> => 
Расширение функциональности HELM 


YAGNI KISS 


HELM PLUGINS 


Helm Diff Preview helm upgrade as a coloured diff 
helm-monitor Plugin to monitor a release and rollback based 
on Prometheus/ElasticSearch query 


helm-s3 Helm plugin that allows to use AWS S3 as a [private] 
chart repository 
helm-git Install charts and retrieve values files from your Git 


repositories 

helm-secrets Plugin to manage and store secrets safely (based on 
sops) 

helm-tanka A Helm plugin for rendering Tanka/Jsonnet inside Helm 
charts. 


List: https://helm.sh/docs/community/related 
Guide: https://helm.sh/docs/helm/helm plugin 


Files 


./mychart 
[— Chart.yaml 1 apiVersion: vl 
2 Kind: Secret 
3 metadata: 
I— charts 4 name: {{ .chart.Name }}-secret 
backend 5 type: Opaque 

ingress-tls 6 data: 

job-load-db-dump 7 token: |- 

job-mc-create-bucket 8 (( .Files.Get "files/configl.conf" | b64enc }} 
job-migrations 

minio 1 

postgres 

postgres-backup 

rabbitmq 
plates 

secretl.yaml == 

_envs.tpl 
values.yaml 


ІГІТІТІТІ 
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Happy helming! 


poke 
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